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IMAGE PROCESSING METHOD AND 
APPARATUS FOR COMPARING EDGES 
BETWEEN IMAGES 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates to an image processing 
method for detecting an object from an input image using a 
template image, and relates to an image processing appara- 
tus therefor. 

2. Description of the Related Art 

Conventionally, a technique is well known in which a 
template image is pre-registered, and the position in an input 
image of an image similar to the template image is detected 
by pattern matching between the input image and the 
template image. However, since distorted perceptions are 
liable to be shaped according to the manner in which the 
background of fhe image similar to the template image is 
formed, Japanese Published Unexamined Patent Application 
No. Hei-5-28273 discloses a technique that has been devel- 
oped to solve this problem. In this publication, a similarity 
value between the template image and the image corre- 
sponding to the template image is defined by the mathemati- 
cal Formula 1. 

Formula 1 

Cv: Correlation coefficient (similarity value) 
M: Number of pixels of template image in x direction 
N; Number of pixels of template image in y direction 
Sx; Derivative value of input image S in x direction 
Sy: Derivative value of input image S in y direction 
Tx: Derivative value of template image T in x direction 
Ty: Derivative value of template image T in y direction 



c,- w 

In detail, an inner product (cos Q) of an angle 0 between 
a normal direction vector of the edge of the template image 
and a normal direction vector of the edge of the input image 
is a component of the similarity value. 

However, there exists a problem in that, as described later 
in detail, if the brightness of a background periphery of an 
image of an object is uneven, the positive and negative 
values of the inner product are reversed. This makes the 
similarity value unsuitable for the real image, and distorted 
perceptions are easily produced, thus making it difficult to 
obtain a desirable recognition result. 

Additionally, the similarity value formula is nonlinear 
with respect to the normal direction vectors of the edges of 
the input and template images, and processing for the 
template image and processing for the input image must be 
performed simultaneously. 

Further, the template image is scanned on the input image. 
A correlated calculation of the input image and the reference 
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image must be performed for each scanning point. There- 
fore, in practicality, a resulting expansion of the amount of 
calculation makes real-time processing impossible. 

5 OBJECTS AND SUMMARY OF THE 

INVENTION 

It is therefore an object of the present invention to provide 
an image processing method and an image processing appa- 
ratus capable of obtaining an accurate, clear recognition 
result and capable of performing high speed processing. 

In an image processing method according to a first aspect 
of the present invention, the amount of matching between a 
template image and an input image is evaluated using a 
similarity value map. An evaluation vector is generated for 
each of the template and input images. The evaluation vector 
includes a component in which a normal direction vector of 
an edge of a specified image undergoes an even number of 
angular transformations. 

With this strucmre, the matching therebetween is properly 
evaluated with no influence on the similarity value even in 
a case in which the positive and negative of an inner product 
(cos 9) of an angle 6 between a normal direction vector of 

25 the edge of the template image and a normal direction vector 
of the edge of the input image are reversed by unevenness 
in the brightness of its background. 

An image processing method according to a second aspect 
of the present invention includes a step of inputting a 

30 specified image for each of a template image and an input 
image and calculating a normal direction vector of an edge 
of the specified image; a step of generating an evaluation 
vector from the edge normal direction vector; a step of 
subjecting the evaluation vector to an orthogonal transfor- 

35 mation; a step of performing a product sum calculation of 
corresponding spectral data for each evaluation vector, 
which has been subjected to ordiogonal transformation, 
obtained for the template image and input image; and a step 
of subjecting a result of the product sum calculation to an 

40 inverse orthogonal transformation and generating a map of 
similarity values; in which a formula of the similarity values, 
the orthogonal transformation, and the inverse orthogonal 
transformation each have linearity. 
Wifli this structure, a Fourier transfonnation value of the 

45 template image and a Fourier transformation value of the 
input image do not need to be simultaneously calculated. In 
otiier words, the Fourier transformation value of the tem- 
plate image is obtained prior to that of the input image, thus 
making it possible to lighten the processing burden and 

50 improve processir^ speed. 

An image processing method according to a third aspect 
of the present invention includes a step of compressing each 
evaluation vector, which has been subjected to the orthogo- 
nal transformation, so as to reduce the processing amount. 

With this structure, what is processed is limited only to an 
effective component (e.g., low-frequency component). 
Thereby processing speed is improved. 

In an image processing method according to a fourth 

50 aspect of the present invention, for the template image, the 
steps taken until the evaluation vector that has been sub- 
jected to the orthogonal transformation is compressed are 
executed before the input image is input, and tiie result is 
stored in a recording unit. 

65 With this structure, processing relating to the template 
image is completed merely by reading from the recording 
unit, and processing speed is improved. 
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Fonnula 1 0 (i.e., the Fourier transfonnation value of tiie similarity value 

Edge normal direction vector of input image L) to an inverse orthogonal transformation unit 11. 

^ The inverse orthogonal transformation unit 11 subj ects the 

!=iVy) Fourier transformation value of the similarity value L to 

I,: Derivative value of input image in x direction ^ "I'^'erse Fourier transformation, and outputs the map L(x,y) 

L: Derivative value of input image in y direction °^ similarity value L to a map processing unit 12. The 

The evaluation vector generation unit 7 inputs the edge Processing unit 12 extracts a high-value point (peak) 

normal direction vector of the input image from the edge SP'^ "^^P ^(^'y)' ^nd outputs its position and its value, 
extraction unit 6, and outputs an evaluation vector of the ,„ The map processmgumt 12 and the construction subsequent 
input image defined by Ihe following two fonnulas. '° ^^^^y ^""^S^ necessary. 

Next, a processing example by the template image of FIG. 
Formula 11 2 will be described with reference to FIG. 4 and FIG. 5. If 

Length normalized vector of input image the input image is as shown in FIG. 4(a), the edge extraction 

unit 6 extracts an edge component in the x direction as 
15 shown in FIG. 4(&), and extracts an edge component in the 
7 y direction as shown in FIG. 4(c). 

J = (Jx. Jr) = — g result of the aforementioned processing, a similarity 

value map L(x,y) shown in FIG. 5(a) is obtained. Herein, the 
front end of the arrow indicated as the "maximum value" is 
Formula 12 ^ °^ °^P' apparent from a comparison 

J * u ..t. t. 1 J /X- c J i\ with the input image of FIG. 5(b), it is understood that the 

When assumed to be a threshold (for fine edge removal), . • • i i • j » i 

_^ V e V; correct pomt is clearly recogmzed at only one point. 

the evaluation vector K of the input image is: In the conventional technique, the calculation must be 

_ performed by the number of times of the product, i.e., 2AB 

' I in order to successively scan the template image on input 

-» images and to obtainfp^lFormula 1 at each position, wherein 

K=(KM=i<^sm. sui(2p))=(2/^-i,2V^) gj^e of the inpurimage is A(=2^) and the size of the 

^j^^ template image is B. Herein, the number of calculations is 
evaluated by the number of times of the product with high 

_ _ calculation costs. 

In contrast, in this embodiment, FFT is performed twice 

The input image processing part 200 differs from the by the orthogonal transformation unit 3 and 8, the product 

template image processing part 100 only in the fact that sum calculation is then performed by the multiplication unit 

normalizing processing of division by "n" is not performed. 10, and inverse FFT is performed once by the inverse 

In other words, evaluation by the even-numbered times orthogonal transformation unit 11. The number of calcula- 

angle, the normalizing processing to 1 in length, and the tions is merely the number of times of the product of 

noise removal processing are performed in the same way as 3{(2y-4)A+4}+2A. 

in the template image processing part 100. In a comparison of the number of calculations therebe- 

Next, a structure subsequent to the evaluation vector tween, the number of calculations of the product according 

generation unit 7 will be described. As shown in FIG. 1, in to this embodiment is about 1/100 of the number of calcu- 

the input image processing part 200, the evaluation vector of lations of the product according to the conventional tech- 

the input image that is output from the evaluation vector nique, if A=256x256=2'^, and B=60x60. As a result, tmusu- 

generation unit 7 is subjected to Fourier fransfoimation by ally high-speed processing is carried out. 

the orthogonal transformation unit 8, and is output to the The discrete correlation theorem of Fourier transforma- 

compression unit 9. tion cannot be used in such a nonlinear formula as Formula 

The compression unit 9 compresses the evaluation vector 1 of the conventional technique. Therefore, in the conven- 

that has been subjected to Fourier transformation, and out- tional technique, processing for the template image cannot 

puts the result to the multiplication unit 10. Herein, the be performed prior to that for the input image as shown in 

compression unit 9 compresses the object to be processed to FIG. 1 of this embodiment. In other words, in the conven- 

the same frequency band as the compression unit 4 (e.g., tional technique, both the template image and the input 

halves on the low frequency side in the x and y directions, image must be processed simultaneously. Also in this 

respectively, in this embodiment). respect, the processing speed in this embodiment is higher 

The compression unit 9 is, of course, omitted when the than that in the conventional technique, 
data amount is small, or when high-speed processing is not 55 /xj v j- * 2'1 

required, and the compression unit 9 is likewise omitted . . , 

when the compression unit 4 is omitted in the template embodmient, a conjugate compression umt 13 and 



image processing part 100. 



a conjugate restoring unit 14 are added to the elements of 



Next, the multiplication unit 10 and other construction ^ f ^l^'t ^ compression u 

- 13 fiirther halves the Founer transformation value of the 



subsequent to this will be described. When the processing 
the template image processing part 100 and in the input 



evaluation vector of the template image that has been read 



hnage processing pa^ 200 is completed, the multiplication ".^ °^ ^"^^^ "'^J^S^^^ 

unit 10 inputs a Fourier transformation value of each evalu- Pi«perUes of Founer transformation, 

ation vector of the template and inputs images from the This respect will be descnbed. The foUowmg formula is 

recording unit 5 and the compression unit 9. 65 estabUshed for the specffum obtained by real number Fou- 

Thereafter, the recoiding unit 5 performs a product-sum- "^^ tJ^sfomiation. 

calculation according to Formula 9, and outputs the result A(u,v)=A(,-u,-vy Fomula 13 
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wherein the size range is a b. 

The similarity value formula that is linear before being 
subjected to Fourier transformation is still linear after being 
subjected thereto. Therefore, the position of the addition unit 
16 can be changed from the First Example to the Second i 
Example, for example. 

Thereby, an object to be processed by the addition unit 16 
is reduced more than in the First Example because the object 
is compressed by compression unit 4. Therefore, processing 
speed is improved. 2 

(Embodiment 4) 

Referring now to FIGS. IS and 16, Embodiment 4 dis- 
closes an efBcient technique for enhancing the value of the 
maximum point of the similarity value map described in 2 
embodiments 1 and 2 into a stronger peak value. 

Generally, in the similarity value map, a peak appears in 
a part overlapping the template image. In this embodiment, 
a peak pattern "p" around and including a maximum point 
is used as a filter for the similarity value map. The value of 
a part similar to the peak pattern in the similarity value map 
is amplified. 

As shown in FIG. 15, in this embodiment, a peak pattern 
processing part 300 is added to the structure of FIG. 1 
showing embodiment 1. 

FIG. 16 shows a mask for this peak pattem. As indicated 
in FIG. 16, in this peak pattern, normalization is made to set 
an average value at 0. 

In the peak pattem processing part 300, an orthogonal 
transformation unit 17 subjects this peak pattem to Fourier 
transformation, and a compression unit 18 compresses a 
Fourier transformation value before recording compressed 
data in a recording unit 19. 

Since a mask is used, the similarity value formula is 
estabUshed not by Formula 8 but by the following formula 
16 that reflects the mask. 

Formula 16 



then performs a product-sum-calculation, and outputs the 
Fourier transformation value of the similarity value cor- 
rected by the peak pattem. 

A peak pattem filter can filter the similarity value map L 
according to the following formula, but, if so, a large amount 
of product sum calculations will be inefBciently needed. 



In contrast, in this embodiment, simple, accurate process- 
ing is performed according to Formula 16 without perform- 
ing a large amount of calculations like Formula 17. 

Therefore, according to this embodiment, the peak point 
of the similarity value map is efficiently amplified. Addi- 
tionally, a part sunilar to the template is clearly and stably 
detected from the input image while reflecting the peak 
pattem. 

(Embodiment 5) 

In this embodiment, a pixel mean in the range of the 
template image as well as the similarity value between the 
edge of the input image and the edge of the template image 
25 is added to a similarity judgment. 

This structure is shown in FIG. 17. Like FIG. IS, a mask 
pattem processing part 400 is added to the stmcture of FIG. 
1 of Embodiment 1 . 
However, the structure of FIG. 17 differs from that of FIG, 
30 15 in that the mask pattem processing part 400 is used, not 
to input a peak pattern, but to input a template image. Also, 
a mask pattern generation unit 20 is provided for generating 
a mask pattem that depends on this image. 
Like FIG. 15, the output of the mask pattern generation 
35 unit 20 is subjected to Fourier transformation by the 
orthogonal transformation unit 21. The result is then com- 
pressed by the compression unit 22, and the compressed data 
is recorded in the recording imit 23. 
Further, since a mask is used, the similarity value formula 
40 is expressed not as Fomiula 8 but as the following formula 
that reflects the mask. 



Vx(«,v) = |] V»(«,v) 



P: Peak pattem low frequency spectrum 
L(u,v): Similarity value before filtering 
M(u,v): Similarity value after filtering 

The multiplication unit 10 reads data from the recording 
unit 5, the recording imit 19, and the compression unit 9, 



eu, y) = y) + ^ 9(/, /«)/(*+/, y + m) 



Q(x,y): Pixel-average-added similarity value 
q(l,m): Mask pattem 
50 L(x,y): Similarity value before filtering 
I(x,y): Input image data 

For the same reason mentioned in Embodiment 4, a lai;ge 
amount of product sum calculations is inefScientiy needed if 
multipUcation is peribnned in this way. 
55 Formula 19 is obtained by subjecting this to Fourier 
transformation, whereby the calculation is performed very 
simply. 

e(«.v)=i(u,v)+^(tt,v)/(u.v) Foimula 19 

60 The multipHcation unit 10 performs a product-sum-cal- 
culation according to Formula 19. 

Next, the relationship between the template image and the 
mask pattem is described with reference to FIG. 18. Herein, 
in order to add a pixel mean in the range of the template 

65 image to a similarity judgment, the mask pattem generation 
unit 20 generates a mask pattem as shown in FIG. lS(b) for 
a template image as shown in FIG. 18(a) 0 
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In greata- detail, in the template image of FIG. 18(a), a 
value of 1/N is set at each point of the inner part (inside ihe 
circle) whose pixel mean is to be calculated, and a value of 
0 is set at all other points. N is the number of points in the 
inner part, and the result of addition of the values of all 5 
points of the mask pattern is 1. 

According to this embodiment, a mean of pixel values 
inside the image can also be added to the similarity value, 
and a specified object is extracted from the input image more 
accurately and more efiSciently. lo 

A mean of a square value of each pixel is calculated such 
(hat data in which each pixel of the input image is squared 
by the input image processing part is formed, and the same 
processing is applied thereto. 

Therefore, a distributed value as well as a mean within a is 
range is eflBciently calculated. 

(Embodiment 6) 

Referring to FIG. 19, Embodiment 6 discloses a technique 
by which a bilaterally symmetric image with respect to a jo 
template image is efficiently processed. In addition to the 
structure of FIG. 1 of Embodmeut 1, a symmetric vector 
generation unit 24 is provided between the recording unit 5 
and the multiplication unit 10, As in Embodiment 1, For- 
mula 8 is used as the similarity value formula in this 35 
embodiment. 

Referring now to FIG. 20(a), a description is given of how 
to treat a bilaterally symmetric template image. For 
example, if the template image of FIG. 20(a) is an original 
image, a template image in which the original image has 30 
been bilaterally reversed is shown as in FIG. 20(6). 

The relationship between the edge normal direction vec- 
tors of these template images is expressed by the following 
formula 20. 

Formula 20 

Edge normal direction vector of image resulting from 
subjecting original template image to bilateral reversal 

40 

T'Ax,y)=Tyi-x.y) 

The evaluation vector of the template image that has been 
bilaterally reversed is expressed by the following formula 
21. 

W=(JK4x,y),?li>ix,y)y. Evaluation vector of 7' 

'y=(f^x(x,y),lj^y')y. Evaiuation vector of 7 
Wxfy,y)=2Tii'-l=2{-Hc(rX.y)f-l=2Tx(,-xj>)^-l=Vx 



example, reversing the positive and negative of the evalu- 
ation vector of the original template image. 

Therefore, the evaluation vector of the template image 
that has been bilaterally reversed is obtained merely by 
allowing the symmetric vector generation unit 24 to apply 
Formula 23 to the evaluation vector of the original template 
image of the recording unit 5 in FIG. 19. 

When simply assumed, there is no need to perform 
complex processing, such as a procedure in which the image 
of FIG. 20(Z>) is generated from the image of FIG. 20(a), and 
the evaluation vector is again calculated from the image of 
FIG. 20(Z>). 

Thereby, the evaluation vector of the template image that 
has been bilaterally reversed is generated without direct 
calculations, and processing speed is improved. Addition- 
ally, recording capacity is saved because the need to pur- 
posely store the template image that has been bilaterally 
reversed is obviated. 
(Embodiment 7) 

Referring now to FIG. 22(b), in embodiment 7, eye/ 
eyebrow extraction processing is added to the face extrac- 
tion processing described in Embodiment 3. Eyes/eyebrow 
candidate range is roughly extracted from the input image of 
FIG. 22(a) according to the processing described in Embodi- 
ment 3. 

Point biserial correlation coefficient filters shown in FIG. 
23(a) through FIG. 23(rf) aie applied to each point of the 
image of this eye/eyebrow candidate range, a map of point 
biserial correlation values is then formed. Points where the 
correlation value in the map are high are set as an eye center 
point 3002 and as an eyebrow center point 3003, respec- 

The point biserial correlation coefficient r\ is defined by 
the following formula (reference; Multivariate Analysis 
Handbook, page 17, Modem Mathematics). 



Concerning Fourier transformation, because of the rela- 
tion of Formula 22, Formula 23 is obtained by applying 
Fommla 22 to Formula 21. 

Ax)=g(.-x)-*Ku)F'gi-«) Formula 22 

»'>.v)=f'A(-«.v) 



Overall range: addition of 1st range to 2nd range 
ni-- Number of pixels of 1st range 
5 n: Number of pixels of overall range 
x7: Average brightness level of 1st range 
x^: Average brightness level of 2nd range 
S: Standard deviation of overall range 
Value in y is constant when mask size is fixed. 
' FIG. 23(a) through FIG. 23(^0, FIG. 23(a) show the 
positional relationships of all ranges. FIG. 23(i) shows an 
overall range mask. FIG. 23(c) shows a 1st range mask. FIG. 
23(d) shows a 2nd range mask. 
. If the filter shape according to this point biserial correla- 
' tion coefficient is formed as shown in FIG. 23(a), it is 
expected that the eyebrow center 3002 and the eye center 
3003 are extracted as shown in FIG. 22(c). 
Next, filter processing according to the point biserial 
J correlation coefficient will be described. 

First, main components of Formula 24 are expre.ssed by 
the following formula 25. 

r,=JJ(x+i.yy)Ml(i.j) 



In detail, the evaluation vector of the template image that 
has been bilaterally reversed is easily generated by, for 



x^U{x+i.y+jW(i.j) 
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x'=i:j{x+i.y+j)M<'(i.J) 

whereinD 



Ml: 1st range mask 
M2: 2nd range mask 
Ma: Overall range mask 
I(x,y): Each pixel value of input image 
The following formulas 26-28 are obtained by subjecting 
each component to Fourier transformation. 

(fF-^)=7(Afi-M2)» Fonnula 26 



x^=jM„* Formula 28 

wherein 

J(x,y)=l\x,y) 20 

In order to perform these processings, the structure of 
FIG. 21 is formed, for example. First, each mask of the 
overall range, the 1st range, and the 2nd range is subjected 
to Fourier transformation by the orthogonal transformation 
unit 51 through S3. 

An input image and a result of the face extraction 
described in Embodiment 3 are input to the eye/eyebrow 
candidate range extraction unit 54 through the map process- 
ing unit 12. Based on these inputs, the eye/eyebrow candi- 
date range extraction unit 54 extracts only the eye/eyebrow 
candidate range shown in FIG. 22(6) from FIG. 22(a). Data 
concerning this eye/eyebrow candidate range is subjected to 
Fourier transformation by an orthogonal transformation unit 
55 without any changes. The result is then squared by a 35 
square unit 56, and is subjected to Fourier transformation by 
an orthogonal transformation unit 57. 

Thereafter, data shown in Formula 27 is input through a 
multiplication unit 58 to an inverse orthogonal transforma- 
tion imit 62 precedent to a r| map forming unit 65. Likewise, 40 
data shown in Formula 26 is input to an inverse orthogonal 
transformation unit 63 through a multiplication unit 60, and 
data shown in Formula 28 is input to an inverse orthogonal 
transformation unit 64 through a multiplication unit 61. The 
input data is subjected to inverse Fourier transformation by 45 
the inverse orthogonal transformation unit 62 through 64, 
and is output to the t] map forming unit 65. 

Thereafter, the t) map forming unit 65 performs a calcu- 
lation according to Formula 24 when receiving the data fijom 
the inverse orthogonal transformation unit 62 through 64, 50 
and outputs a map ii(x,y) of the point biserial correlation 
coefBcient. 

The eye/eyebrow center extraction unit 66 extracts two 
points having high values from the map Ti(x,y) output by the 
Ti map forming unit 65, and outputs them as the eyes center 55 
and an eyebrow center, respectively. 

In this structure, multiplication must be performed 
roughly 1 5x 1 5xN=225N times, wherein "15x1 5" (pixels) is 
the filter size of FIG. 23(a), and N is the number of pixels 
of the input image. 60 

In contrast, according to this embodiment, the number of 
product calculations is roughly N+[(2y-4)N+4]x5+12N+N+ 
N=5N(2y-1)+20 times. For convenience, the calculation of 
"/' is assumed to be equal to one multiplication processing. 

That is, (calculation amount of this embodiment)<(calcu- 65 
lation amoimt of the conventional technique) under the 
condition that y is 22 or less. 



Usually, a huge image in which N is 222 or more is not 
used. Therefore, it is understood that the image processing 
of this embodiment is smaller in the number of calculation 
times than that of the conventional processing, and is 
performed faster than that of the conventional processing. 

Thus, according to this embodiment, the point biserial 
correlation coefBcient filter processing is more efficiently 
performed. 

The positions of other face parts, such as eye |comersj 
) mouth alges, nostrils, and irises, are calculated by variously 
changing the masks shown in FIG. 23(a) through FIG. 23(rf). 

(Embodiment 8) 
This embodiment discloses a technique for expanding the 
. function of the face extraction according to Embodiment 3 
and extracting a mouth range, which is a facial organ, from 
a &ce image. 

As described in Embodiment 3 with reference to FIG. 14, 
a mouth candidate range shown in FIG. 25(b) is extracted 
I from the input image of FIG. 25(a). 

When the extracted mouth candidate range is projected 
onto the Y-axis (i.e., when the total h of pixel values is 
plotted along the X-axis), a graph roughly shown in FIG. 



S(c) is 

The total h is defined by the following formula 29. 



I(x,y): Each pixel value of input image 
w: Width of mouth candidate range 
h(x,y): Projection value 

In order to efEciently obtain this total by use of orthogonal 
transformation, a mask shown in FIG. 25(<i) is prepared. 
Formula 29 is rewritten Hke the following formula 30, 
including this mask. 



m: Mask 

When this is subjected to Fourier transformation, the 
following formula 31 is obtained. 

ri{u,v)=m{ii,v)*i{u,v) Fonnula 31 

That is, this projection value is obtained by subjecting the 
input image and the mask pattern to Fourier transformation, 
thereafter performing the calculation of Formula 31, and 
subjecting the result to inverse Fourier transformation. 
Thereby, the map "h(x,y)" of the projection value "h" is 
obtained. 

From the above result, it is desirable to form the structure 
of FIG. 24, for example. As shown in FIG. 24, the iiqjut 
image and the mask pattem are subjected to Fourier trans- 
formation by the orthogonal transformation unit 25 and 26, 

respectively. 

Thereafter, the transformation value is multiplied by the 
multiplication unit 27, and is subjected to inverse Fourier 
transfomiation by the inverse orthogonal transformation unit 
28. As a resuh, the map h(x,y) of the projection value is 
obtained. 
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conection ftmction determimng unit 37 ou^uts the correc- 
tion ftinction to the image correction unit 34. 

The horizontal axis of the brightness histogram of FIG. 30 
indicates brightness values. It will be understood that only a 
part of the dynamic range that can express brightness is 5 
used. Therefore, the correction fimction determining unit 37 
determines the correction fimction so as to |filly| use the 
dynamic range, and thereby the image correction unit 34 can 
correct the face image so that the image is easily discerned, 
or, more specifically, so that it can have abundant gradations ic 
in appearance. As a result of this correction, tlie face image 
is infallibly easily viewable although there might be a 
situation in which a part other than the face image is 
lightened or darkened. 

Herein, since the correction function is determined on the 15 
basis only of the facial irmer image, correction is never 
misled by a non-face part. Therefore, after correction, it is 
guaranteed that viewability of the face is improved. 

Although FIG. 30 shows the drawing so as to make 
correction only to the cut-out face image, similar correction 20 
is made to the whole input image. 

Further, as a feature to be sought by the image feature 
extraction unit 36, an index that represents the lightness/ 
darkness of an image, such as brightness, is used as 
described above. In addition, a chroma average or a hue 25 
average may be used. 

If the chroma average is used, the correction fimction 
determining unit 37 can output, for example, a chroma 
amplification coefficient to the image correction unit 34. If 
the hue average is used, the correction fijnction determining 30 
unit 37 can output, for example, a hue rotational angle to the 
image correction unit 34. 

Herein, a pale face image is more vividly corrected when 
the chroma average is used as a feature. When the hue 
average is used, a less reddish face image is corrected to be 35 
more reddish. In either case, the fece image is made more 
natural. Additionally, a combination of two or more of 
brightness, chroma average, or hue average may be used as 
a feature. 

Having described preferred embodiments of the invention 40 
with reference to the accompanying drawings, it is to be 
understood that the invention is not limited to those precise 
embodiments, and that various changes and modifications 
may be effected therein by one skilled in the art without 
departing from the scope or spirit of the invention as defined 45 
in the appended claims. 

What is claimed is: 

1. An image processing method comprising the steps of: 
inputting a specified image for a template image; 

inputting a specified image for an input image; 50 
calculating an edge normal direction vector of said speci- 
fied image; 

generating an evaluation vector from said edge normal 
direction vector; 

normalizing said evaluation vector of said template image 55 
by the number of edge normal direction vectors; 

subjecting said evaluation vector to orthogonal transfor- 
mation; 

performing a product sum calculation of corresponding 

spectral data for each evaluation vector that has been 60 

subjected to orthogonal transformation; 
subjecting a result of said product sum calculation to 

inverse orthogonal transformation and generating a 

map of similarity values; and 
a formula of said similarity values, said orthogonal trans- 65 

formation, and said inverse orthogonal transformation 

each have linearity. 
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2. Hie image processing method of claim 1, flirther 
comprising the step of: 

compressing each evaluation vector that has been sub- 
jected to orthogonal transformation so as to reduce a 
processing amount. 

3. The image processing method of claim 1, wherein for 
said template image, the steps taken until said evaluation 
vector that has been subjected to orthogonal transformation 
is compressed are executed before said input image is input, 
and storing results thereof. 

4. The image processing method of claim 1, fiirther 
comprising the step of: 

normalizing said evaluation vector with respect to a 
vector length. 

5. The image processing method of claim 1, fiirther 
comprising the steps of: 

reducing a data amount usmg complex conjugate proper- 
ties of orthogonal transformation before performing a 
product sum calculation; and 

restoring said data amount after performing said product 
sum calculation. 

6. An image processing method comprising the steps of: 
inputting a specified image for a template image; 
inputting a specified image for an input image; 
enlarging/reducing said template image to various sizes; 
calculating an edge normal direction vector of said speci- 
fied image; 

generating an evaluation vector from said edge normal 
direction vector; 

subjecting said evaluation vector to orthogonal transfor- 
mation; 

subjecting said evaluation vector of each size to addition 
processing; 

performing a product sum calculation of corresponding 
spectral data for each evaluation vector that has been 
subjected to orthogonal transformation; 

subjecting a result of said product sum calculation to 
inverse orthogonal transformation and generating a 
map of similarity values; and 

a formula of said similarity values, said orthogonal trans- 
formation, and said inverse orthogonal transformation 
each have linearity. 

7. The image processing method of claim 6, wherein, for 
said template image, said addition processing of said evalu- 
ation vector is carried out after executing said step of 
compressing each evaluation vector so as to reduce the 
processing amount. 

8. The image processing method of claim 1, wherein said 
template image is an image of a typified face. 

9. The image processing method of claim 1, fiirther 
comprising the steps of: 

preparing a peak pattern that makes a peak of said 

similarity value steep; and 
subjecting data of said peak pattern to orthogonal trans- 

fonnation to said product sum calculation. 

10. The image piocessmg method of claim 1, further 
comprising the steps of: 

forming a mask pattern that depends on said template 
image; and 

subjecting data of this mask pattern to orthogonal trans- 
formation to said product sum calculation. 

11. The image processing method of claim 10, wherein 
said mask pattern includes an average of a number of pixels 
in an image of said template image. 
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12. The image processing method of claim 1, fiirflier orthogonal ttansfonnation and has been obtained for 
comprising the steps of: said template image and said input image; 

for said template image, processing positive and negative an addition unit operable to perform addition processing 

signs of said evaluation vector of said original template of said evaluation vector of each size; and 

image; and 5 an inverse orthogonal transformation unit operable to 

generatinganevaluation vector of a bilaterally symmetri- subject a result of said product sum calculation to 

cal image with respect to said original template image, inverse orthogonal transformation and further operable 

by which said generated evaluation vector is applied to to generate a map of similarity values; 

said product sum calculation. said evaluation vector including a component in which an 

13. ITie image processing method of claim 8, further to edge normal direction vector of a specified image 
comprising the steps of: undergoes even-numbered times angular transfonna- 

generating a map ofpointbiserial correlation coefBcients tion, and a formula of said similarity values, said 

on the basis of an extracted face image; and orthogonal transformation, and said inverse orthogonal 

responsive to said correlation coefficients, calculating a transformation each have linearity, 

position of said face part. 15 18. The image processing apparatus of claim 17, wherein 

14. The image processing method of claim 8, further said template image processing part includes a recording 
comprising the steps of: unit operable to record said evaluation vector that has been 

calculating a distribution of projection values in a y-di- compressed to reduce a processing amoimt and that has been 
rection on the basis of said extracted fece image by use subjected to orthogonal transformation, and a result obtained 
of said mask pattern; 20 by compressing said evaluation vector that has been sub- 
calculating two maximum points from said distribution; J'ected to orthogonal transformation is stored in said record- 

and ing unit before inputting said input image, 

ou^utting a range between said two maximum points as '"^S^ processing apparatus of claim 17, further 

a mouth range. comprising: 

15. The image processing method of claim 8, further 25 a conjugate compression unit between said recording unit 
comprising the steps of: and said multipHcation unit; 

dividing said input image into onlv said face image and ^^^d conjugate compression miit operable to reduce the 

parts other than said face imag^ on the basis of said ^ ''^^^P^^^ conjugate properties of 

extracted fece image; orthogonal transformation; , . . 

embedding a digital watennark only into said face image; '° « '^''^^^^ ^^^nng umt between said multiplication umt 

combining said face image into which said digital water- ^. ^^'^ mv^rse orthogonal transformation umt; said 

mark^sbeenembeddtdwithpartsotherthLsaidface conjugate restoring umt operable to restore the data 

image to produce a combined result; and 1 ^fj^ f ^ '^.'^P^'^ 

^ °. ., L- J ix properties of orthogonal transformation, 

outputtmgsaidcombmedrosuh 20 The image processing apparatus of claim 17, wheiein 

16. The mage processing method of clami 8, further ^^j^ ^^^^^ ^ ^p^^^^ ^ p^„^ ^^^^ 
compnsmg esepso. processing of said evaluation vector of said template image 

dividing said input mnage mto only said face image and ^^^^ compressing said vector so as to reduce the processing 

parts other than said face image on the basis of said amoimt 

extracted fece image; 21. The image processing apparatus of claim 17, fiarther 
editmg only said face image; comprising a peak pattern processing unit operable to sub- 
combining said face image after editing with parts other -^^^ ^ pggk pattern by which a peak of a similarity value is 

than said face image to produce a combined result; are ^lade steep to orthogonal transformation and compress said 

outputtmg said combmed result. peak pattern that has been subjected to said orthogonal 

17. An image processing apparatus comprising: 45 transformation so as to reduce the processing amount, 
a template image processing part operable to input a wherein a result obtained by subjecting data of this peak 

template image and calculate an edge normal direction pattern to said orthogonal transformation is applied to a 

vector of said template image, normalize said edge product sum calculation of said multiplication unit, 

normal direction vector of said template image, gener- 22. The image processing apparatus of claim 17, further 

ate an evaluation vector from said normalized edge 50 comprising: 

normal direction vector, subject said evaluation vector a mask pattern processing part operable to form a mask 

to orthogonal transformation, and compress said evalu- pattern that depends on said template image and gen- 

ation vector that has been subjected to said orthogonal erate data obtained by subjecting data of this mask 

transformation so as to reduce the processing amoun<r| pattern to orthogonal transformation and by compress- 

an input image processing part operable to input an input 55 ing it, wherein a processing result of said mask pattern 

image and calculate an edge normal direction vector of processing part is applied to a product sum calculation 

said input image, generate an evaluation vector from of said multiplication unit. 

said edge normal direction vector, subject said evalu- 23. The image processing apparatus of claim 22, wherein 

ation vector to orthogonal transformation, and com- said mask pattern includes a mean of a number of pixels 

press said evaluation vector that has been subjected to m inside an image of said template image, 

said orflxogonal transformation so as to reduce the 24. The image processing apparatus of claim 17, further 

processing amount; comprising: 

an enlargement/reduction unit operable to enlarge or a symmetric vector generation unit operable to process 

reduce said template image to various sizes; positive and negative signs of said evaluation vector of 

a multiplication unit operable to perform a product sum 65 an original template image recorded in said recording 

calculation of corresponding spectral data about each unit, and flirther operable to generate an evaluation 

evaluation vector that has been subjected to said vector of a bilaterally symmetric image with respect to 
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said original template image, wherein said evaluation 
vector generated by said s)anmetric vector generation 
unit is applied to a product sum calculation of said 
multiplication unit. 

25. The image processing apparatus of|l7, fiiither com- 5 
prising a map forming unit operable to form a map of a point 
biserial correlation coefficient on the basis of an extracted 
face image, and an extraction unit operable to calculate a 
position of a face part from the formed map. 

26. The image processing apparatus ofjl7, further com- lo 
prising a maximum point extraction unit operable to calcu- 
late a projection value distribution in a y direction by use of 

a mask pattern on the basis of an extracted face image, and 
ftirther operable to calculate two maximum points from this 
distribution, and ou^utting a range between said maximum 1 5 
points such as a mouth range. 

27. The image processing app laratus of|l7, further com- 
prising: 

a face image cutting-out unit operable to separate an input 
image into only a face image and parts excluding said 20 
face image on the basis of an extracted face image; 

a digital watermark embedding unit operable to embed a 
digital watermark only into the face image; and 

an image synthesizing unit operable to combine said face 
image into which said digital wateimaik has been 
embedded with parts excluding said face image and 
outputting the combined data. 

28. The image processing apparatus of|l7, further com- 
prising: 

a face image cutting-out unit operable to separate an input ^° 
image into only a face image and parts excluding said 
face image on the basis of an extracted face image; 

an image correction unit operable to edit only said fece 
image; and 

an image synthesizing unit operable to combine an edited 
face image with parts excluding said fece image and 
outputting them. 



29. The image processing method of claim 8, flirdier 
comprising: 

cutting out a face image from said input image on the 
basis of an extracted face image; 

extracting a facial inner image from said fece image that 
has been cut out; 

calculating a feature that correct said face image on the 
basis of said extracted face image; 

determining a correction function on said basis of said 
obtained feature; and 

applying image correction based on said determined cor- 
rection fimction at least onto said fece image that has 
been cut out. 

30. The image processing method of claim 29, wherein 
said feature is a combination of at least two of brightness, 
chroma average, and hue average. 

31. The image processing apparatus of|l7, further com- 
prising: 

a face image cutting-out unit operable to cut out a fece 
image from said input image on a basis of an extracted 
face image; 

a face internal range extraction unit operable to extract a 
facial inner image from said fece image that has been 

cut out; 

a image feature extraction unit operable to calculate a 
feature that serves to correct said face image on a basis 
of said extracted face image; 

a correction function determining unit operable to deter- 
mine a correction fimction on a basis of said obtained 
feature; and 

a image correction unit operable to apply image correc- 
tion based on said determined correction ftinction at 
least onto said face image that has been cut out, 

32. The image processing apparatus of claim 31, wherein 
said feature is a combination of at least two of brightness, 
chroma average, and hue average. 



